Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(downloaders): replace outdated baraag downloader #171

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

blackphillip-hkp
Copy link
Contributor

Replaces the current broken baraag downloader with one that works as of 2024-10-03.

@floogulinc
Copy link
Collaborator

Does this use the mastodon API?

@blackphillip-hkp
Copy link
Contributor Author

blackphillip-hkp commented Oct 3, 2024

@floogulinc Yep, with a similar lookup strategy as the Fanbox downloader using the Fanbox API (actually used it and the ExH downloaders as inspiration). I tested this downloader with over 100 queries downloading over 50,000 status attachments over the last 24 hours without any errors or any of baraag's infamous rate limits except a handful of unresolvable "media proxy" attachment links.

I also skipped over the included "mastodon parsers" and built custom parsers just for baraag, but I'm pretty sure these artist id lookup and status attachments parsers could be applied to any mastodon site.

Seems like a downside of connecting the mastodon API and Hydrus is there will always be one ignored request. I'm new to building downloaders, and while it seems pretty straightforward maybe there's a way to avoid that extra request that I'm not thinking of.

@floogulinc
Copy link
Collaborator

I've had a generalized mastodon/Pleroma API parser in the discord for a while now which includes baraag. I just haven't added it here yet because I was waiting on a feature that was only added to hydrus in the latest version to be able to properly handle getting notes of the post content without just dumping the html into the note. You might want to check that out too, a lot of parser work just happens in the discord now rather than here.

@blackphillip-hkp
Copy link
Contributor Author

Neat, I'm not a big discord user but I'll check it out :) I'm currently tweaking the EH/ExH downloader, I've been having some issues with it too, but this might be due to an accidental double-import on my part. It reports something like Cannot parse zzz - renamed due to auto-import - Could not find a parser for zzz - renamed due to auto-import - so-and-so URL Class!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants